Hardware realizations of neural networks cover a wide spectrum of electronic circuits from digital emulators to fully analog networks. The so-called neural networks are based on parallel computing architectures which emulate neural functions such as pattern recognition and character recognition.
Digital implementations of neural networks provide a more flexible framework than analog circuits for achieving computational accuracy with arbitrary precision. On the other hand, analog circuits theoretically permit a much higher density of connections per chip than the corresponding digital circuits. An analog implementation has been described in U.S. patent application Ser. No. 654,998 (H. P. Graf Case 7). Even hybrid implementations have appeared recently in which a mixture of analog and digital circuitry is employed to realize the neural network.
The neural network includes a plurality of computational elements or neurons each of which perform a variety of mathematical functions on the input signals, images, or other stimuli being processed by the neural network. Each neuron comprises a plurality of synapse circuits in which an input is multiplied by a particular weight. The weighted inputs from the synapse circuits are summed together in the neuron and are processed by a nonlinear squashing function to produce a neuron output An example of a neuron or computational element has been shown and described in U.S. Pat. No. 5,067,164.
Recent realizations of neuron synapse circuits have appeared in hybrid form wherein digital weights are multiplied by analog input signals using multiplying digital-to-analog converters. The digital weight is converted to analog form and multiplies the analog input signal to produce the analog synapse output. For example, two quadrant multiplication of a digital weight and an analog input is described in Neural Information Processing Systems 2, pp. 769 et seq. (D. S. Touretzky ed. 1990) whereas four quadrant multiplication of the same signals is described in IEEE. J. Solid-State Circuits, Vol. 26, No. 12, pp. 1894-1901 (December 1991).
For these realizations, digital weights are stored in cyclic shift registers or dedicated latch circuits and the analog input signals are supplied to the multiplying digital-to-analog converter. The motivation for this approach is apparently two-fold. First, digital storage is relatively inexpensive. Second, since the weights remain constant for a particular neuron while the input signals are changing during operation of the neuron, the weights are transformed into digital form for storage in the digital memory or registers associated with the multiplying digital-to-analog converter in the neuron. This approach requires separate digital storage for each weight in conjunction with the multiplying circuit for each synapse circuit.